1. Field of the Invention
The present invention relates to an edge enhancement process and system for image data representing static or moving (i.e., video) images.
2. Description of the Related Art
Luminance transient improvement or enhancement refers to the sharpening of an image by steepening edge transitions on the luminance component of the image, thereby increasing the original signal bandwidth. There are two general approaches to making an edge steeper. One approach increases pixel values on the high side of an edge and decreases pixel values on the other side of the edge, thereby making a gradual transition more abrupt. The other approach is to replace pixel values near the edge with pixel values further away from the edge.
In general, existing transient improvement circuits are based on one of three transient improvement techniques. The first technique uses edge detectors. Usually, the center of the edge is located and the degree of enhancement applied to a given pixel depends upon the distance away from the center of the edge and also on the magnitude of the edge. International Patent Application Publication No. WO 01/67392, entitled System and Method for Improving the Sharpness of a Video Image, and incorporated herein by reference in its entirety, describes an edge sharpening system including a sub-pixel edge center locator that reduces visible jitter after enhancement. The system also provides gain control for the enhancement signal using the relative position of edge information to tackle the “over-the-hill” problem that occurs for an edge when pixels of a second edge are shifted.
The second technique employs a strong peaking filter. Although peaking filters do not increase frequency components, they steepen the edges. By clipping the overshoot and undershoots, high frequency components are introduced to increase the original signal bandwidth and sharpen the image at the same time. In addition, special boundary conditions are detected at the pixel level whereby adjustments are constrained.
The third technique extracts the maximum and minimum values of a local window. U.S. Pat. No. 6,094,205, entitled Sharpness Control, and incorporated herein by reference in its entirety, performs edge enhancement using the 2nd derivative to select the minimum or maximum pixel value and then generates a fading transition between the original and the selected minimum/maximum pixel value using the magnitude of the first derivative.
International Patent Application Publication No. WO 01/52188, entitled Method and Apparatus for Edge Detection, and incorporated herein by reference in its entirety, discloses a method for edge detection using a low pass second derivative for increased robustness in noise and aliasing, a thresholded first derivative to distinguish between noise and edges, and a third to first derivative comparison as an edge frequency calculation to differentiate soft and hard edges for suitable edge enhancement.
There are a number of difficulties with existing edge enhancement processes. The first has been attributed to discrete time sampling in that an edge center may not coincide with the pixel grid. If the resulting error in the exact location of the edge becomes too large, jitter becomes visible after enhancement. Generally, this problem only occurs when edge enhancement is based on the distance from the edge center that is identified by the pixel grid.
A second problem is the over-enhancement of soft and hard edges that results in an artificial-looking image. An example of soft edges is the enhancement of facial features. The 3D information of the visualized object, represented on the 2D plane, is lost as the smooth edges are straightened and steepened. In addition, visible contouring of images is introduced after edge steepening because the subset of pixel representation for the edge has been reduced significantly. Furthermore, when hard edges are steepened further, unnatural looking plateaus on both sides of the edge are generated.
Thirdly, edge processing methods are generally not robust to noise and often result in images with enhanced noise. In methods based on detected edge centers, local processing masks used with 2nd derivative operators or Laplace operators locate edges most precisely, but are very sensitive to noise. Stochastic gradient operators, on the other hand, are most robust to noise, but are not easy to implement as the level of noise changes. Similarly, methods using peaking filters are very sensitive to noise. The conventional counteraction strategy is to use low pass filters or derivatives, as described in International Patent Application Publication No. WO 01/52188.
A fourth problem is the inflexibility to handle close and complicated edges. Generally in pixel level processing, special boundary conditions are made to cater for such situations but these may not be sufficient to cover all cases. A typical artifact is the result of shifting pixels of another edge and thus changing the contents of the image. As described in International Patent Application Publication No. WO 01/67392, this “over-the-hill” problem can be alleviated by controlling the gain of the enhancement signal based on the distance between the detected edge pixels.
Another problem is the enhancement of fuzzy edges, resulting in obvious jagged edges. Fuzzy edges often occur on a soft focused background and/or image. They may be present due to randomness introduced on edges for special effects purposes or the deteriorating quality of old films.
Finally, edge enhancement brings about an aliasing side effect in the vertical direction that manifests itself in the form of saw-tooth edge. The horizontal steepening of a diagonal edge draws pixel values further from the edge nearer to the center. As the edge slopes away from the vertical, the further spaced apart the center of the 1D edge relative to its counterpart above and below the line, the more obvious the saw tooth pattern becomes to the observer. This effect worsens and appears as horizontal flickering lines when the image is interlaced and the edge is moving.
It is desired therefore, to provide an edge enhancement system and process that alleviate one or more of the above difficulties, or at least to provide a useful alternative.